ci: run Zephyr builds in parallel #286
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
No longer useful for any purpose. Signed-off-by: Luca Burelli <l.burelli@arduino.cc>
Save the prepared build environment as an artifact after the initial setup, to allow for multiple jobs to reuse it without repeating the initial Zephyr configuration steps. Signed-off-by: Luca Burelli <l.burelli@arduino.cc>
Convert the sequential building and packaging of all board binaries into a matrix job, allowing parallel execution. This makes the workflow more efficient and independent of the number of supported boards. Signed-off-by: Luca Burelli <l.burelli@arduino.cc>
Use a common function to handle logging messages for GitHub Actions and for the console, for a cleaner output. Signed-off-by: Luca Burelli <l.burelli@arduino.cc> :1
These changes allow the packaging and testing jobs to start even if some of the build jobs have resulted in errors, thus maximizing the amount of testing that can be performed in parallel CI workflows even if one board is causing issues. The build step artifacts will be saved with a "failed-" prefix in case of errors and can be inspected for further information. Signed-off-by: Luca Burelli <l.burelli@arduino.cc>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request refactors the GitHub Actions workflow for packaging the core, splitting the build pipeline into more granular jobs, improving artifact handling and cleanup, and enhancing error reporting in packaging scripts.
The main goals are to improve parallelization and provide clearer diagnostics during packaging:
The workflow is split into distinct jobs:
build-env(fetches the Zephyr environment only once),build-board(builds board-specific binaries in parallel),package-core(packages each core artifact in paraallel), andtest-core(tests each board in parallel). This change now makes the whole CI pipeline basically independent of the number of boards / packages being tested.CI now tries to run as much as possible in case an error is detected, while still indicating which parts of the CI build are failing. Build failures for individual boards are summarized and included in the GitHub Actions step summary, and the full contents of the build directory can be retrieved for easy inspection. Tests are attempted even when some of the boards did not compile successfully.
See here for a sample run that includes a failed Giga build.
Future work could move the Zephyr env preparation step using Docker; I think that should speed up the process compared to the current artifact upload/download.